Discriminative Training of Language Model

We show how discriminative training methods, namely the Maximum Mutual Information and Maximum Discrimination approach, can be adopted for the training of N-gram language models used as clas-siiers working on symbol strings. By estimating the model parameters according to a discriminative objective function instead of Maximum Likelihood, the emphasis is not put on the exact modeling of each class, but on the right classiication of the samples. The methods are shown to be suited for a variety of applications, such as the recognition of regulatory DNA sequences and language identiication. Using phonotactic information, we achieve an error reduction of 10.7% (phoneme sequences) or 41.9% (code-book classes) with respect to the standard ML estimation on a corpus of English and German sentences.